Search for: All records

The ensemble method is a promising way to mitigate the overestimation issue in Q-learning, where multiple function approximators are used to estimate the action values. It is known that the estimation bias hinges heavily on the ensemble size (i.e., the number of Q-function approximators used in the target), and that determining the 'right' ensemble size is highly nontrivial, because of the time-varying nature of the function approximation errors during the learning process. To tackle this challenge, we first derive an upper bound and a lower bound on the estimation bias, based on which the ensemble size is adapted to drive the bias to be nearly zero, thereby coping with the impact of the time-varying approximation errors accordingly. Motivated by the theoretic findings, we advocate that the ensemble method can be combined with Model Identification Adaptive Control (MIAC) for effective ensemble size adaptation. Specifically, we devise Adaptive Ensemble Q-learning (AdaEQ), a generalized ensemble method with two key steps: (a) approximation error characterization which serves as the feedback for flexibly controlling the ensemble size, and (b) ensemble size adaptation tailored towards minimizing the estimation bias. Extensive experiments are carried out to show that AdaEQ can improve the learning performance than the existing methods for the MuJoCo benchmark. 

We produce an explicit description of the K-theory and K-homology of the pure braid group on n strands. We describe the Baum–Connes correspondence between the generators of the left- and right-hand sides for n = 4. Using functoriality of the assembly map and direct computations, we recover Oyono-Oyono’s result on the Baum–Connes conjecture for pure braid groups [24]. We also discuss the case of the full braid group on 3-strands. 

This paper studies distributed Q-learning for Linear Quadratic Regulator (LQR) in a multi-agent network. The existing results often assume that agents can observe the global system state, which may be infeasible in large-scale systems due to privacy concerns or communication constraints. In this work, we consider a setting with unknown system models and no centralized coordinator. We devise a state tracking (ST) based Q-learning algorithm to design optimal controllers for agents. Specifically, we assume that agents maintain local estimates of the global state based on their local information and communications with neighbors. At each step, every agent updates its local global state estimation, based on which it solves an approximate Q-factor locally through policy iteration. Assuming a decaying injected excitation noise during the policy evaluation, we prove that the local estimation converges to the true global state, and establish the convergence of the proposed distributed ST-based Q-learning algorithm. The experimental studies corroborate our theoretical results by showing that our proposed method achieves comparable performance with the centralized case. 

Multimetallic nanoclusters (MMNCs) offer unique and tailorable surface chemistries that hold great potential for numerous catalytic applications. The efficient exploration of this vast chemical space necessitates an accelerated discovery pipeline that supersedes traditional “trial-and-error” experimentation while guaranteeing uniform microstructures despite compositional complexity. Herein, we report the high-throughput synthesis of an extensive series of ultrafine and homogeneous alloy MMNCs, achieved by 1) a flexible compositional design by formulation in the precursor solution phase and 2) the ultrafast synthesis of alloy MMNCs using thermal shock heating (i.e., ∼1,650 K, ∼500 ms). This approach is remarkably facile and easily accessible compared to conventional vapor-phase deposition, and the particle size and structural uniformity enable comparative studies across compositionally different MMNCs. Rapid electrochemical screening is demonstrated by using a scanning droplet cell, enabling us to discover two promising electrocatalysts, which we subsequently validated using a rotating disk setup. This demonstrated high-throughput material discovery pipeline presents a paradigm for facile and accelerated exploration of MMNCs for a broad range of applications. 

Lightweight, flexible, and electrically conductive thin films with high electromagnetic interference (EMI) shielding effectiveness are highly desirable for next‐generation portable and wearable electronic devices. Here, spin spray layer‐by‐layer (SSLbL) to rapidly assemble Ti₃C₂T_xMXene‐carbon nanotube (CNT) composite films is shown and their potential for EMI shielding is demonstrated. The SSLbL technique allows strategic combinations of nanostructured materials and polymers providing a rich platform for developing hierarchical architectures with desirable cross‐functionalities including controllable transparency, thickness, and conductivity, as well as high stability and flexibility. These semi‐transparent LbL MXene‐CNT composite films show high conductivities up to 130 S cm⁻¹and high specific shielding effectiveness up to 58 187 dB cm²g⁻¹, which is attributed to both the excellent electrical conductivity of the conductive fillers (i.e., MXene and CNT) and the enhanced absorption with the LbL architecture of the films. Remarkably, these values are among the highest reported values for flexible and semi‐transparent composite thin films. This work could offer new solutions for next‐generation EMI shielding challenges.